Airport gate visual docking guidance system digital twin

ABSTRACT

Airport gate visual docking guidance system digital twin methods, systems, and devices are described herein. One method includes: identifying that a device of the visual docking guidance system is non-responsive, determining which device is non-responsive, initiating start-up of one visual docking guidance system program selected from: a visual docking guidance system software program, a central computing system program, or an operating station program, selecting a digital twin data set corresponding to the device of the visual docking guidance system that was non-responsive, wherein the set includes parameter data that was customized by a user of the visual docking guidance system and is particular to the device of the visual docking guidance system that was non-responsive, and sending the digital twin parameter data to the device of the visual docking guidance system that was non-responsive to update one or more parameters stored in association with the visual docking guidance system program.

TECHNICAL FIELD

The present disclosure relates to an airport gate visual dockingguidance system digital twin.

BACKGROUND

Airports have increasingly been adopting systems generally referred toas visual docking guidance systems (VDGS) to manage aircraft on theground and gate area functions. Each visual docking guidance systemunit/device includes an sensor data processing unit, a dockingcontroller, a display unit, and at least one video, RADAR, and/or LIDARsensor.

Typically, one visual docking guidance system is deployed per airportgate. Central computing system software connects to the dockingcontroller to determine the status of the gate and to send dockingschedules and manual control commands to the gate area.

One concern in a visual docking guidance system is that the ability tobring the visual docking guidance system up and running are very timeconsuming tasks. This can occur, for example, in case of system failuresand replacement of visual docking guidance system units, in case ofsystem software upgrades, or regular software maintenance activities

There are various reasons that this may occur, for instance: a loss ofconfiguration of various visual docking guidance system sub-systems whenthe system fails or there might be records/details manually maintainedin the system, but these details are not updated when certain parametersare changed in a sub-system. Further, the configuration data of each ofthe sub-systems and rules defined for gate operations are typicallyrequired for the central computing system to maintain communication withvisual docking guidance systems and perform normal functionality.

Even though redundant servers are available for central computing systemsoftware, some configuration and calibration information related to avisual docking guidance device or unit is not maintained in the centralcomputing system software as those details are very specific to visualdocking guidance system sub-systems. Further, in certain airports, dueto cost constraints, they do not utilize redundant serves. In that case,any failure in the central computing system will lead to huge downtimeif the central computing system is not restored quickly.

Assuming that there is a scenario where visual docking guidance systemat a particular gate is not working and that the visual docking guidancesystem hardware needs to be completely replaced, after changing to a newvisual docking guidance system hardware unit, a technician must bringback the system to the same normal state that it was before byconfiguring all necessary things properly for that gate. The technicianmust be clear on what the configuration of the system was before itfailed and ensure it is brought back to the same state. This process andaccountability can be difficult and time-consuming tasks and may lead todown time of the gate. This is an additional cost to airport operations.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a component view of a visual docking guidance systemaccordance with one or more embodiments of the present disclosure.

FIG. 2 illustrates a schematic view of a visual docking guidance systemin accordance with one or more embodiments of the present disclosure.

DETAILED DESCRIPTION

In the present disclosure, a visual docking guidance system digital twinstructure is proposed which can be executed in the central computingsystem or in a separate on-premise (airport) computing device that pullsout the various sub-system state and configuration details and createsan information data model of the system and maintains it for speedyrecovery. The information data model includes, all system configurationdetails, network topology, application configuration details, and stateinformation of each of the visual docking guidance system units, thecentral computing system, and operator station. The applicationconfiguration details can, for example, include the details related tocommunication and networking details, digital certificates, airportstand layout three dimensional (3D) configurations, stop barconfiguration, association of supported aircraft model, docking workflowalgorithm configuration for each visual docking guidance system unit,obstacle detection rules, wing tip collision rules, stand centerlineadjustment parameters, sensor calibration and/or adjustment parameters,gate system equipment configurations, digital 10 pin configurations,NTP/master clock server configurations, and/or other customized settingsthat may be lost when the visual docking guidance system unit isreconfigured.

Whenever some sub-systems of a visual docking guidance system fail orreplacement of some visual docking guidance system units takes place, anoperator will select the previous visual docking guidance system unit'sdigital twin via the digital twin agent and initiate a command torestore the parameters from the selected digital twin. The digital twinsystem will upload the necessary details to bring the system to normaloperation.

In prior art systems, this may take several hours which can be reducedto a few minutes through use of the embodiments of the presentdisclosure. Each visual docking guidance system sub-system (e.g., visualdocking guidance system units, the central computing server, operatorstations) can also have digital twin agent software which can help incapturing these configuration updates whenever there is a change in thesystem state.

For example, in some embodiments, at least one of the visual dockingguidance system units, the central computing server or operator stationsincludes a second digital twin agent. This second digital twin extractsdata that is to be part of the digital twin parameter data set on thefirst digital twin agent from the device, server, or unit on which thesecond digital twin agent resides.

The second digital twin agent can also assist in installing theparameter data. For example, the second digital twin agent can includecomputer executable instructions to receive the digital twin parameterdata set from the first digital twin agent and install the digital twinparameter data set on the device, server, or unit on which the seconddigital twin agent resides. The executable instructions can be stored inmemory and executed by a processor of the device, server, or unit onwhich the second digital twin agent resides.

The information can be protected using encryption and digital signaturestechnologies so that, at-rest, this information will not be tamperedwith by any malicious users. The overall digital twin of each sub-systemcan be represented as a model and this model can be either maintained inan on-premise server or in a remote server (e.g., cloud) if the airportallows data to be maintained remotely.

This solution provides an extra advantage by replacing a faulty and/orlegacy (i.e., being phased out due to the device no longer beingupgradable) physical device with a new hardware device without muchcommissioning effort, with quick turnaround time, and/or low cost.Embodiments of the present disclosure also enable remote commissioningfrom a remote service location after replacing the hardware componentson the premises.

A visual docking guidance system unit includes a large display providedat the gate that provides a variety of information to the groundcrew/flight crew workers at that particular gate. For example, someinformation types can include information from cameras and sensors atthe gate area that detect aircraft to identify position, movement,orientation, and/or aircraft type characteristics and this informationcan be communicated on the display as well.

At some airports all of the visual docking guidance system units areconnected to a central computing system (typically contained on acentral server located at the airport). The function of the centralcomputing system is to convey information about the layout of theairport, the layout of the gates, and the movement of aircraft on theairside maneuvering paths.

The operator can, for example, give directions to pilots in the variousaircraft, can give instructions to ground crew, and can assign aircraftto specific gates, if not already assigned. The operator can also changevisual docking guidance system unit configurations via changingoperational parameters of the configuration used by a particular visualdocking guidance system unit by routing such parameter changes, made bythe operator at the operator station, through the central computingsystem, to the particular visual docking guidance system unit.

The central computing system has several functionalities. For example,it controls a number of docking controllers, image capture devices, andsensors that are part of a visual docking guidance system units and actsas a communicative connection between the operator stations, visualdocking guidance system units, and the digital twin host.

Information that may be provided by the digital twin host includesconnectivity configuration information between a particular visualdocking guidance system unit and the central computing system. Forexample, Internet credentials, Internet Protocol information, and otherconnectivity parameters can be some information that can quickly beprovided, if a visual docking guidance system unit fails or is replaced.

The digital twin can also provide operator station configurationinformation, such as IP address, security information, certificates,name and date information, and current system status, in someembodiments. Also can monitor the certificate expiry dates and provideearly alerts.

To populate this information in the digital twin, the operator selects,via the operator station, the types of items to be maintained in thedigital twin host and then that information is retrieved and copied fromits current location along with data regarding where the informationcopied is to be placed in relation to the new unit software once arecovery from a failure or replacement of a unit has occurred.

The copied information is then retrieved and copied again, periodically,to update the twin with the most current information. In this manner,recovery from a failure or unit replacement can bring the system asclose to operating the way it was before the failure or replacement, ascan be and as soon as it can be accomplished.

The twin is not an exact copy of the visual docking guidance systemsoftware being recovered, but rather saved customized, operationalparameters and computing device executable instructions to locate where,in memory, each parameter is to be placed within the visual dockingguidance system software. Accordingly, the recovery process would firstrestart the current version or load the updated version of the visualdocking guidance system software, the operator would select a visualdocking guidance system unit digital twin parameter data set toimplement at a particular visual docking guidance system unit, and thenthe digital twin agent will be executed to modify the data of theparticular visual docking guidance system software that corresponds tothe selected digital twin.

In some embodiments, this configuration data can be reused for similargates with similar configuration data features. This can result inreduced modifications of the similar gates needing updates, which helpsin airport expansion with new gates and other situations, such as, whereupdates to multiple similar gates is needed.

In the following detailed description, reference is made to theaccompanying drawings that form a part hereof. The drawings show, by wayof illustration, how one or more embodiments of the disclosure may bepracticed.

These embodiments are described in sufficient detail to enable those ofordinary skill in the art to practice one or more embodiments of thisdisclosure. It is to be understood that other embodiments may beutilized and that process, computerized, and/or structural changes maybe made without departing from the scope of the present disclosure.

As will be appreciated, elements shown in the various embodiments hereincan be added, exchanged, combined, and/or eliminated so as to provide anumber of additional embodiments of the present disclosure. Theproportion and the relative scale of the elements provided in thefigures are intended to illustrate the embodiments of the presentdisclosure and should not be taken in a limiting sense.

The figures herein follow a numbering convention in which the firstdigit or digits correspond to the drawing figure number and theremaining digits identify an element or component in the drawing.Similar elements or components between different figures may beidentified by the use of similar digits. For example, 112 may referenceelement “12” in FIG. 1 , and a similar element may be referenced as 212in FIG. 2 .

As used herein, “a” or “a number of” something can refer to one or moresuch things. For example, “a number of gate areas” can refer to one ormore gate areas.

FIG. 1 illustrates a component view of a visual docking guidance systemaccordance with one or more embodiments of the present disclosure. Asillustrated, an airport can have many gate areas 104, where aircraft 102individually park to embark and disembark passengers and flight crew,refuel, resupply, clean the aircraft, load and unload cargo, and otherfunctions between flights.

A visual docking guidance system unit 106 assists in providinginformation to the grounds crew workers and/or to the flight crew. Thisincludes flight information, such as flight number and airline andarrival time, and can provide real-time guidance to the flight crew forproper placement of the aircraft within the gate area.

In embodiments of the present disclosure, the visual docking guidancesystem units are communicatively connected to a central computing system108 that allows operators at a number of operator stations to monitorand provide directions/instructions/information to the various aircraft102, gate areas 104, and visual docking guidance system units 106.

The operator stations include a computing device with processor andmemory and software accessed thereon for monitoring aircraft, gateareas, and/or visual docking guidance system units. Typically, thesedevices include a user interface (computer monitor) and a user input(keyboard, touch screen display). This allows the operator to senddirections/instructions to the aircraft, update information presented onthe visual docking guidance system unit display, update the visualdocking guidance system software or make changes to parameters thereof,among other functions.

As mentioned above, the visual docking guidance system units can includeone or more cameras and/or sensors for guiding the aircraft to thecorrect position and orientation within the gate area and/or can alsoidentify the aircraft type. In some implementations, these camerasand/or sensors may have specific parameters set by the operator that, ifa visual docking guidance system unit were to go offline, the dataregarding these parameters would be lost. Examples of parametersinclude, by are not limited to: camera direction, camera tilt, camerazoom, point cloud density, calibration parameters, and other parameterdata.

The system 100 also includes a digital twin host 110. The digital twinhost can include a digital twin agent that is a software program thatexecutes the digital twin process. The digital twin host can also storedata regarding parameters to be sent to a visual docking guidance systemunit, the central computing system 108, and/or the operator stations112, as will be discussed in more detail with respect to FIG. 2 .

In some implementations, the data can be stored at a remote location ona computing device, such as a cloud server 114. In some embodiments, theremote computing device can be synced with the data on the host 110 toprovide an up to date back up of the digital twin data. Further, in someembodiments, the central computing system can have one or more redundantbackup servers or the software can be distributed on multiple devices toreduce downtime if the primary central computing system goes offline orhas some other issue.

If a visual docking guidance system unit or operator station unit failsor is replaced, the digital twin agent can be informed of the particularvisual docking guidance system unit or operator station unit that needsdigital twin data. An operator then selects that particular unit asneeding digital twin data. The digital twin agent then searches andlocates the data for the identified unit and sends the data to thecentral computing system 108 for routing to the identified unit. In someembodiments, the digital twin agent can also send installation softwarewith the data. The installation software includes executableinstructions for placing particular pieces of data in correct locationsin memory related to the parameters of the identified unit.

FIG. 2 illustrates a schematic view of a visual docking guidance systemin accordance with one or more embodiments of the present disclosure. Inthe system shown in FIG. 2 , the system 220 includes a number of visualdocking guidance system unit 206, a central computer 208, and anoperator station 212.

The visual docking guidance system units each include docking controllerparameters including status information and configuration information(cfg). The visual docking guidance system units each also can, forexample, include sensor data processing unit status information andconfiguration information, manual control board information, and anaircraft model database.

The docking controller includes the operation parameters for the displayand the information presented on the display. Parameters can includesize, color, and orientation of the presented content, and content typesthat are selected to be presented among other parameters.

The sensor data processing unit can be a sensor/camera that capturesstill images, video, RADAR, LIDAR, or other imaging techniques. Asdiscussed above, parameters can include, but are not limited to tilt,zoom, pan, field of view, point density, etc.

This parameter data can be periodically loaded, via the centralcomputing system, to a storage area 222 for visual docking guidancesystem parameter data in the digital twin host computing device 210. Inthis manner, the parameters are ready for quick recovery when needed dueto a failure of a visual docking guidance system unit or a replacementof a unit.

Likewise, in some embodiments, parameters of the central computingsystem 208 can be provided to the digital twin to be used upon failureor replacement of a central computing system unit. As illustrated inFIG. 2 , this can, for example, include docking rules parameters,general sensor calibration files, visual docking guidance system unitconnectivity configuration parameters, operator station configurationparameters, aircraft model configuration parameters, stand layoutconfiguration parameters, and other runtime state or configurationparameters. These parameters can, for example, be stored in their ownmemory location 224 in the digital twin host 210.

Further, in various embodiments, parameters of the one or more operatorstations 212 can be provided to the digital twin, via the centralcomputing system, to be used upon failure or replacement of an operatorstation unit. This can, for example, include display parameters,operator preference parameters (e.g., for layout of informationpresented to the operator on the display of the operator station), andother runtime state or configuration parameters (e.g., alert states andactions, security parameters, etc.). These operator station parameterscan, for example, be stored in their own memory location 226 in thedigital twin host 210.

Additionally, in some implementations, the digital twin parameter datacan be stored in a remote location on a computing device 214. In someembodiments, the digital twin computing server can be virtual orphysical in nature. In this example, each visual docking guidance systemunit has its parameters stored in a different memory storage location232-1 through 232-N, with data for the parameters of the centralcomputing system stored at 234 and the operator stations stored at 236.

Although specific embodiments have been illustrated and describedherein, those of ordinary skill in the art will appreciate that anyarrangement calculated to achieve the same techniques can be substitutedfor the specific embodiments shown. This disclosure is intended to coverany and all adaptations or variations of various embodiments of thedisclosure.

It is to be understood that the above description has been made in anillustrative fashion and not a restrictive one. Combination of the aboveembodiments, and other embodiments not specifically described herein,will be apparent to those of skill in the art upon reviewing the abovedescription.

The scope of the various embodiments of the disclosure includes anyother applications in which the above structures and methods are used.Therefore, the scope of various embodiments of the disclosure should bedetermined with reference to the appended claims, along with the fullrange of equivalents to which such claims are entitled.

In the foregoing Detailed Description, various features are groupedtogether in example embodiments illustrated in the figures for thepurpose of streamlining the disclosure. This method of disclosure is notto be interpreted as reflecting an intention that the embodiments of thedisclosure require more features than are expressly recited in eachclaim.

Rather, as the following claims reflect, inventive subject matter liesin less than all features of a single disclosed embodiment. Thus, thefollowing claims are hereby incorporated into the Detailed Description,with each claim standing on its own as a separate embodiment.

1.-20. (canceled)
 21. A method for recovering a device of a visualdocking guidance system, comprising: initiating a start-up of one visualdocking guidance system program on a device in the visual dockingguidance system that was non-responsive, the program selected from: avisual docking guidance system program, a central computing systemprogram, or an operating station program; selecting, from a number ofdigital twin parameter data sets, a digital twin parameter data set thatis particular to the device of the visual docking guidance system thatwas non-responsive; and sending the selected digital twin parameter dataset to the device of the visual docking guidance system that wasnon-responsive to update one or more parameters stored in associationwith the visual docking guidance system program.
 22. The method of claim21, wherein the method includes populating at least one of the number ofdigital twin parameter data sets by selecting a number of types of itemsto be included in the at least one digital twin parameter data set priorto initiating the start-up of one visual docking guidance system programon a device that was non-responsive.
 23. The method of claim 22, whereinthe method further includes identifying where the items are to be placedin relation to software on the device of the visual docking guidancesystem that was non-responsive or new device software once a replacementof the non-responsive unit has occurred.
 24. The method of claim 22,wherein populating at least one of the number of digital twin parameterdata sets by selecting a number of types of items to be included in theat least one digital twin parameter data set, includes populatingmultiple digital twin parameter data sets such that each data setcorresponds to a different visual docking guidance system.
 25. Themethod of claim 22, wherein populating at least one of the number ofdigital twin parameter data sets by selecting a number of types of itemsto be included in the at least one digital twin parameter data set,includes populating multiple digital twin parameter data sets such thateach data set corresponds to a different operator station unit.
 26. Themethod of claim 22, wherein populating at least one of the number ofdigital twin parameter data sets by selecting a number of types of itemsto be included in the at least one digital twin parameter data set,includes selecting at least one item type for a visual docking guidancesystem unit from the types including: docking controller state data,docking controller configuration data, sensor data processing unit statedata, and sensor data processing unit configuration data.
 27. The methodof claim 22, wherein populating at least one of the number of digitaltwin parameter data sets by selecting a number of types of items to beincluded in the at least one digital twin parameter data set, includesselecting at least one item type for a central computing system devicefrom the item types including: docking rules data, sensor calibrationdata, visual docking guidance system unit connectivity configurationdata, and operator station configuration data.
 28. The method of claim21, wherein at least one of the number of digital twin parameter datasets includes state information of each of the visual docking guidancesystem units, the central computing system, and operator station. 29.The method of claim 21, wherein at least one of the number of digitaltwin parameter data sets includes at least one of: digital certificates,airport layout, algorithm configuration for each visual docking guidancesystem, obstacle detection rules, wing collision rules, middle lineadjustment parameters, camera calibration, camera adjustment parameters,sensor calibration, and sensor adjustment parameters.
 30. The method ofclaim 21, wherein at least one of the number of digital twin parameterdata sets includes at least one of: data selected from the data typesincluding: system configuration details, network topology, andapplication configuration details.
 31. The method of claim 21, whereinat least one of the number of digital twin parameter data set includesdata related to connectivity configuration information for communicationbetween two of: the central computing system device, and a visualdocking guidance system unit or the operator station unit.
 32. A digitaltwin device for recovering a device of a visual docking guidance system,the digital twin device having a processor and memory, wherein thememory has instructions executable by the processor to: select, from anumber of digital twin parameter data sets, a digital twin parameterdata set that is particular to the device of the visual docking guidancesystem that was non-responsive; and send the selected digital twinparameter data set to the device of the visual docking guidance systemthat was non-responsive to update one or more parameters stored inassociation with the visual docking guidance system program.
 33. Thedevice of claim 32, wherein the digital twin device includes computerexecutable instructions that send at least one of the number of digitaltwin parameter data sets after a start-up of one visual docking guidancesystem program selected from: a visual docking guidance system softwareprogram, a central computing system program, or an operating stationprogram, is initiated.
 34. The device of claim 32, wherein the digitaltwin device further includes computer executable instructions thatperiodically send at least one of the number of digital twin parameterdata sets that was customized by a user of the visual docking guidancesystem to a storage area within the memory prior to identifying that adevice of the visual docking guidance system is non-responsive.
 35. Adigital twin system for recovering a device of a visual docking guidancesystem, the digital twin system comprising: a visual docking guidancesystem, comprising: a number of visual docking guidance system units; acentral computing server communicatively connected to the visual dockingguidance system units; and a number of operator stations communicativelyconnected to the central server; and a digital twin devicecommunicatively connected to the central server wherein the digital twindevice includes a digital twin agent and a number of digital twinparameter data sets that are particular to a device of the visualdocking guidance system that was non-responsive; and the digital twindevice sending at least one of the number of digital twin parameter datasets selected to the device of the visual docking guidance system thatwas non-responsive to update one or more parameters stored inassociation with the visual docking guidance system program.
 36. Thesystem of claim 35, wherein the visual docking guidance system units,the central computing server and operator stations are located at anairport facility and wherein the digital twin system further includes aremote device that is communicatively connected to the digital twindevice and receives at least one of the number of selected digital twinparameter data sets from the digital twin device.
 37. The device ofclaim 35, wherein the operator station includes a computing device withprocessor and memory and software accessed thereon for monitoringaircraft, gate areas, and/or visual docking guidance systems.
 38. Thedevice of claim 37, wherein the operator station further includes adisplay that presents at least one of the number of digital twinparameter data sets that is particular to the device of the visualdocking guidance system.
 39. The device of claim 38, wherein at leastone of the number of digital twin parameter data sets is accessed viathe display of the operator station and customized by a user of thevisual docking guidance system.
 40. The device of claim 37, wherein thecomputing device within the operator station includes instructions tosend at least one of the number of digital twin parameter data setscustomized by the user of the visual docking guidance system to astorage area within the digital twin device.